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ABSTRACT 



As part of its Andrew W. Mellon Foundation-funded Future of Scholarly Communication 
Project , the Center for Studies in Higher Education (CSHE) has hosted two meetings to explore 
how peer review relates to scholarly communication and academic values. In preparation for an 
April 2010 workshop, four working papers were developed and circulated. They are presented as 
drafts here. (The proceedings from the April 2010 meeting will be published at a future date.) 
The topics covered include assessing the myriad forms peer review takes in the academy, which 
forms of peer review are used for which specific academic purposes (e.g., tenure and promotion, 
publishing, extramural funding, national and international stature), the considerable costs to 
universities in subsidizing the entire peer review process through faculty salaries, and the 
perception that, although peer review represents the best available system, there are nonetheless 
a multitude of problems with it, including its inherent conservatism. 

The topics of the working papers are: (1) Peer Review in Academic Promotion and Publishing: 
Norms, Complaints, and Costs, (2) New Models of Peer Review: Repositories, Open Peer 
Review, and Post Publication Metrics, (3) Open Access: Green OA, Gold OA, and University 
Resolutions, and (4) Creating New Publishing and Peer Review Models: Scholarly Societies, 
Presses, Libraries, Commercial Publishers, and Other Stakeholders. 

Mirroring our work published in Assessing the Future Landscape of Scholarly Communication: 
An Exploration of Faculty Values and Needs in Seven Disciplines (Harley et al. 2010), we 
conclude that there is a need for a more nuanced academic reward system that is less dependent 
on citation metrics, slavish adherence to marquee journals and university presses, and the 
growing tendency of institutions to outsource assessment of scholarship to such proxies. Such a 
need is made more urgent given the challenges to institutional review of assessing 
interdisciplinary scholarship, new hybrid disciplines, the rise of heavily computational sub- 
branches of disciplines, the development of new online forms of edition-making and 
collaborative curation for community resource use, large-scale collaborations, and multiple 
authorship. Compounding the problem further is the insidious and destructive “trickle down” of 
tenure and promotion requirements from elite research universities to less competitive 
institutions and the mounting pressure from governments, often unrealistic, on scholars in 
developed and emerging economies alike to publish their research in the most select outlets. The 
overall global effect is a growing glut of low-quality publications that strains the efficient and 
effective practice of peer review, legitimate academic publishing endeavors, and library 
acquisition budgets. More nuanced and capacious tenure and promotion criteria at research 
universities might lead to a neutralization of the currently unsustainable publishing “arms race.” 
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WORKSHOP ON PEER REVIEW: OVERVIEW AND BACKGROUND 



As part of its Andrew W. Mellon Foundation-funded Future of Scholarly Communication Project , the 
Center for Studies in Higher Education (CSF1E) has hosted two meetings to explore how peer review 
relates to scholarly communication and academic values. 1 The topics covered included assessing the 
myriad forms peer review takes in the academy, which forms of peer review are used for which specific 
academic puiposes (e.g., tenure and promotion, publishing, extramural funding, national and international 
stature), and the considerable costs to universities in subsidizing the entire peer review process through 
faculty salaries. Additionally, a considerable amount of time was spent discussing the perception that, 
although peer review represents the best available system, there are nonetheless a multitude of problems 
with it, including its inherent conservatism. There was also discussion, which mirrors our work published 
in Assessing the Future Landscape of Scholarly Communication: An Exploration of Faculty Values and 
Needs in Seven Disciplines (Flarley et al. 2010), of the need for a more nuanced academic reward system 
that is less dependent on citation metrics, slavish adherence to marquee journals and university presses, 
and the growing tendency of institutions to outsource assessment of scholarship to such proxies. Such a 
need is made more urgent given the challenges to institutional review of assessing interdisciplinary 
scholarship, new hybrid disciplines, the rise of heavily computational sub-branches of disciplines, the 
development of new online forms of edition-making and collaborative curation for community resource 
use (Waters 2007, 2009), large-scale collaborations, and multiple authorship (Kennedy 2003). 

Our discussions also noted the insidious and destructive “trickle down” of tenure and promotion 
requirements from elite research universities to less competitive institutions. For example, some teaching 
institutions are now requiring two books for promotion in the humanities or too many journal articles in 
the sciences. Compounding this problem further, is the mounting pressure from governments, often 
unrealistic, on scholars in developed and emerging economies alike to publish their research in the most 
select outlets, ostensibly to determine the distribution of governmental funds and/or to meet national 
imperatives to achieve research distinction internationally. The overall global effect is a growing glut of 
low-quality publications that strain the efficient and effective practice of peer review, legitimate academic 
publishing endeavors, and library acquisition budgets. Perhaps if more nuanced and capacious tenure and 
promotion criteria were made explicit at research universities, it could provide a pragmatic “signaling 
effect” to other institutions and government ministries. Such a change might lead to a neutralization of the 
unsustainable publishing “arms race.” 

Flowing from our discussions was also a proposal that it would be useful to examine the implications of 
separating publication from the peer-review process, and vice versa. How might that be most effectively 
and practicably accomplished given the currently entrenched system of peer review (which is organized 
primarily by publishers but carried out by faculty)? Not surprisingly, the motivations expressed in our 
discussions for pursuing such an inquiry are multiple, and reflect the various interests of stakeholders 
such as faculty, university administrators, librarians, publishers, and the public. Perhaps the most 
compelling reason for trying to separate peer review from actual publication behaviors is to facilitate the 
development of alternative means of publication that still meet the academic criteria for the adequacy of 
peer review but are not held hostage by rapacious publishers. Presenting faculty with the possibility of a 
wider array of acceptable alternatives to publish their research than is currently accepted by tenure and 
review committees (and are currently provided by traditional publishers) might just create an environment 
where the purchasing power of cash-strapped libraries is increased and the free flow of ideas is better 
supported. It might also make peer review more uniform and effective across different publishing venues. 
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In preparation for the April 2010 workshop, four working papers were developed and circulated. They are 
presented as drafts in this document. The proceedings from the April 2010 meeting will be published at a 
future date. (The Appendix lists participants in both meetings.) 

The four working papers were developed using the following significant resources: 

• The large volume of relevant published and unpublished material and interviews from the current 
Assessing the Future of Scholarly Communication project (Harley et al. 2010). 

• The recent publication. Assessing the Future Landscape of Scholarly Communication: An 
Exploration of Faculty Values and Needs in Seven Disciplines (Harley et al. 2010). 

• The University as Publisher: Summary of a Meeting Held at UC Berkeley (Harley 2008). 

• The summary of transcripts from the March 5, 2009 CSHE meeting on peer review (described 

above). We shamelessly and liberally plagiarized many ideas presented by attendees at that 
meeting. 

• Web, literature, and other research (e.g., controversies about peer review, publishing costs, 

analysis of successes and failures of various open access resolutions such as those at Harvard, 

MIT, UMCP, and so on). 

We are making these draft working papers available to the larger public as a courtesy. We suspect they 
contain errors, omissions, and copy editing problems. The authors take responsibility for any errors. 

We recommend strongly that readers consult the more formally published documents cited above for 
important background and context to the arguments presented here. 



(1) PEER REVIEW IN ACADEMIC PROMOTION AND PUBLISHING: NORMS, 
COMPLAINTS, AND COSTS 



Introduction: What do we mean by peer review? 

The importance of peer review, also known as scholarly refereeing, flows from being the primary avenue 
of quality assessment and control in the academic world. Peer review has many forms and loci. It acts to 
signal the quality of a piece of work, but also functions as a form of gatekeeping to regulate the entry of 
new ideas into scholarly fields; it “serves to maintain overall standards as well as to recognize individual 
excellence” (Becher and Trowler 2001: 61). The diversity and flexibility of peer review to adapt to 
disciplinary and subdisciplinary needs, while maintaining generally high standards, is its strength 
(cf. Kling and Spector 2004). 2 Moreover, it regulates opportunities throughout a scholar’s career in that it 
attaches strongly to reputation and signals a scholar’s value in a competitive academic marketplace. 
Although the forms of peer review can have different purposes, a scholar may in fact be peer reviewed by 
a relatively small number of people relative to the total amount of peer review received. 

It is useful to distinguish among the many forms that peer review can take: 

• Developmental peer review 

Scholars solicit feedback on work-in-progress from informal networks (e.g., laboratory 
discussions, sharing drafts with colleagues, blogs). 

• Pre-publication peer review 

Scholars present and circulate drafts and more polished work at invited talks, symposia, and 
various sized conferences, as well as post on personal websites and repositories. 3 (An additional 
level of peer review regulates at which venues a scholar can even present.) 
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• Publication-based peer review 

The multiple dissemination outlets for scholarly work (e.g., books, journal articles, conference 
proceedings, edited volumes) undergo different types of formal peer review. Different editorial 
and peer review models include: single- or double-blind peer review, student-edited journals in 
law, prestigious invited contributions in the humanities, “communicated” papers in the 
Proceedings of the National Academy of Sciences, or “lightly” reviewed edited volumes. 

• Post-publication peer review 

Indicators of the significance and impact of a scholar’s work include: book or performance 
reviews, letters to the editor, later citations (particularly in literature reviews) and citation counts, 
author-meets-critics conference sessions, article or book prizes, journal clubs, etc. These 
indicators offer indications as to how a scholar’ s work is received in the field. 

• Peer review of data and other scholarly products 

In some fields, peer review is a central criterion to judge other scholarly products such as 
databases, documentary films, websites, software, etc. 4 

• Institutional peer review in tenure and promotion cases 

In tenure and promotion decisions, peer review is conducted by institutional representatives, as 
well as by external referees who are solicited for letters of support. There is a variety of criteria 
used to judge a successful scholar in a tenure and promotion case in most research universities: 
publication, service, and teaching. Excellence in the latter two holds little weight without a stellar 
publication record and evidence that a scholar’s work: is widely read, is judged to be of high 
quality by internal and external reviewers, and advances the field. 

• Peer review for grants/funding 

Review at this stage evaluates a scholar’ s preliminary ideas and determines if he or she will be 
able to carry out a research program (e.g.. National Institutes of Health 2008, Lamont 2009). 

• Cumulative peer review 

Peer review is often cumulative, in that some forms of peer review inform others. 5 Career work is 
evaluated for superlative prizes, awards, and election to elite societies. 

Publication-based peer review as it relates to institutional review 

As Abbott (2008) describes, around the turn of the 19 th century, informal strategics for manuscript control 
gave way to the professional publication-based peer-review system. The consolidation of formal peer 
review and publication venue has led to the latter becoming a “proxy” for the level of peer review it 
carries out. In tenure and promotion reviews at competitive universities, the emphasis on publishing in 
these top-tier outlets is well documented (e.g., Boyer 1997, Becher and Trowler 2001, Harley et al. 
2010). 6 Over reliance on imprimatur is a type of “outsourcing” peer review by linking the quality, 
relevance, and likely impact of a piece of work to the symbolic brand of its publisher (including the 
publication’s impact factor). 

There is, however, some flexibility built into how a scholar coming up for tenure and promotion is 
judged; “quality over quantity” is a common refrain. This includes giving individual portfolios and 
published work a great deal of in-house scrutiny, 7 increasing the component of “campus review” 
(judgments by individuals in the university) rather than relying so heavily on external letters and citation 
indices, looking to secondary indicators in the absence of large numbers of high-impact publications 
(such as awards or other signs that a scholar’s work has received unique recognition), accepting 
alternative publication formats (e.g., journal articles in lieu of books) in some cases. 

Nevertheless, the emphasis on publication in the top peer-reviewed outlets is growing, not decreasing. In 
addition, there is the trickle-down effect (described above) in which some institutions worldwide (via 
research assessment exercises) are requiring higher levels of peer-reviewed work in the top outlets. And 
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senior scholars expect young scholars to meet the same levels of peer review and certification that they 
faced. 8 Consequently, most young scholars do not risk publishing in outlets that lack prestige; they follow 
the lead of their mentors and place enormous value in outlets with established reputation and prestige. 

What is publication-based peer review good for and why is it valued? 

In the current publishing system, the editorial and peer review of a manuscript takes three factors into 
account: the technical quality of a paper (e.g., it is not “pseudo-science”), its impact and significance in 
the field, and the publishing scope of the journal or press series. Recent quantitative studies have found 
that the large majority of authors surveyed believed that peer review is important, and very few are 
dissatisfied with the peer-review system used by journals (Mark W are Consulting Ltd. 2008, Brown 
2009). 9 This dominant system is viewed as providing many valuable functions: 

Registration and preservation 

• By inserting work into the scholarly record, peer-reviewed publication certifies and guarantees a 
scholar’s intellectual property of research and ideas. 

• Although technology has enabled an increase in the facility of communicating scholarly work 
informally, a formal “copy of record” is necessary for bibliographic purposes. 10 

• Journal publishers and subscribing libraries take responsibility for archiving and preserving 
scholarly work. 

• The physical publication (particularly the book in the humanities) is an important marker of a 
well- and appropriately-crafted argument. 

Quality improvement 

• The peer review process, especially double-blind review, ideally improves the quality and value 
of published work through changes and checks. 11 As noted below, however, it is far from 
foolproof. 

• Senior scholars, acting as editors or reviewers, can play an important role in helping younger 
scholars shape their ideas in subsequent revisions. 

Filtration 

• The book and journal-based peer-review system remains paramount as a filter separating serious 
scholarship from ephemera and poorly conducted research (Harley et al. 2007, 2010, Friedlander 
2008). There is a perception that the Internet has enabled a large amount of poor scholarship to 
proliferate; peer review is seen as an imperfect antidote to that by helping scholars to eliminate a 
certain amount of skepticism in what they read. 12 

• Page limits help scholars keep up to date in a time-efficient manner by narrowing down the vast 
amount of work to that which is of the highest quality and importance. 

Community knowledge production 

• A journal or publishing program frames the development and scope of knowledge in a field. 13 

• The peer-review process establishes and develops fields by creating peers that can work together 
to advance new knowledge or the application of new techniques (Abbott 2001). 

• Many journals, particularly those run by scholarly societies, serve to circulate reviews of 
research, conference information, bibliographies, and other information in the field. 
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Complaints about publication-based peer-review 

As Abbott (2008) notes, complaints about peer review are as old as the process itself. The historic trust in 
and reliance on the reputation of prestige publications as a proxy for the work they disseminate generally 
overshadows consideration of the false positives and false negatives created in the peer-review process. In 
particular, some have argued that the significance of a piece of work is not always correctly predicted by 
the immediate judgment on it in the peer-review process (e.g., Becher and Trowler 2001: 62, 
Shulenburger 2001, Casati et al. 2007). The editor(s) of a scholarly publication also have significant (and 
often unchecked) power in the peer-review process (cf.. Weller 2001). 

Speed and delay costs 

• There are long lag times for formal peer review in many fields. 14 

• There can be a repeated review and rejection of papers as they descend the “journal hierarchy” 
until they are finally accepted (Alberts et al. 2008). 

• The delay cost of time spent in the review and revision process can be costly to the author as well 
as the scholarly community. It delays the recognition of individual work, 15 and its use by the 
scholarly community. Extensive revisions may not always lead to a better publication, and can 
take an author’s time away from developing new research (Casati et al. 2007). 

• The process delays the return on the public investment in research (Jennings 2006). 

• The international profile of submissions to leading journals is also changing, including a large 
volume of papers from emerging economies. The ballooning number of submissions places 
immense burdens on the review process, not only by the need to deal with an increased volume of 
submissions, but by demanding both increased vigilance about quality and veracity, 16 and 
managing language and editing challenges. 

Conservatism, maintenance of orthodoxies, and blocking innovation 

• Peer review can be a double-edged sword, maintaining and reinforcing orthodoxies in academic 
disciplines. In particular, it often fails to seek out, identify, and reward transformative ideas. 17 
This inherent conservatism needs to be continuously managed for the greater good of 
scholarship. 18 

• It is particularly difficult to locate qualified generalists to evaluate work in new, emerging, 
interdisciplinary, and multidisciplinary research areas (Lee 2006). 

• Editors (particularly younger scholars) may be reluctant to rule against the opinions of senior 
reviewers. 

• Controversial or difficult work can be much harder to publish than relatively “unexciting work,” 
because referees may doubt its veracity or require much in the way of revisions. There is so much 
competition for publication in some fields that “one little misplaced comment and you’re out.” 

Difficulties obtaining both enough reviewers and unbiased quality reviews 

• Traditionally, the “peer” referees are not always the literal peers of the author, but are generally 
established scholars with mastery of a particular area who are able to assess the value of new 
work. Consequently, senior scholars receive a disproportionate number of requests to review 
(anecdotally ten or more a month in some cases), and if they agree to review, this only attracts 
more requests. These top scholars are very often overloaded by the increasing number of requests 
and “pass the buck” to other scholars who may be less qualified to referee the submission in 
question (Robeyns 2008). 

• Technical barriers in some countries might prevent participation by local scholars as referees. 
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• It is difficult to find unbiased reviewers on topics of scholarly contention or in small or emerging 
fields. 

• In some fields, there is a fear that reviewer-competitors may “scoop” or delay submitted 
manuscripts for their own personal gain. 

• The blind review process can be “abused and abusing.” Complaints include reviewers demanding 
citations of their work, delivering personal attacks (Resnik et al. 2008), discriminating against 
women and foreign authors, 19 and demanding extensive revisions without bearing their cost (the 
reviewer as “lazy co-author” problem). 20 

• The lack of formal acknowledgement for referees may lead to hurried, superficial reviews. 21 

• The agreement among peer reviewers is not much better than chance. 22 

• There arc only two (and sometimes possibly three) referees for a publication, and many referees 
have complained of a lack of guidance from editors on how to review (Brown 2009). 

Editorial gatekeeping 

• Editorial quality is perceived to be sliding in some fields. Junior editors can be poorly trained. 
Professional editors (or non-practicing scholars) may be unable to make decisions about the 
substance of peer reviews. 23 And scholar-editors may lack the training, time, and resources 
necessary to edit a top-tier scholarly publication. 

• Editors may use their position for personal reward, such as self-publishing their own work (Davis 
2008), or pushing through articles of students or friends. 

• Editors can be biased and favor certain perspectives in “their” journals. Scholars, especially 
young scholars, may need to seek out journal editors “sympathetic” to their research agenda. 

Problems with fraud and validation 

• Ultimately, detecting plagiarism and fraud might be a noble aim of peer review, but is not 
practical. While a recent survey found that a majority of scholars think that peer review should 
detect plagiarism (81%) or fraud (79%), fewer (38% and 33%) think it is actually capable of this 
(Brown 2009). There are many high profile incidents where shoddy, fabricated, and/or 
plagiarized research has been retracted by the top outlets. 24 

• As Becher and Trowler (2001: 63) point out, published research findings are assumed to be 
correct, and further scholarship builds upon them. New work is only published if it builds on 
published ideas, rather than replicates them. Consequently, validation is more or less haphazard, 
until something goes wrong and an error is identified in earlier work. 

• Peer reviewers cannot always ensure the reliability of published findings. Time and lack of access 
to primary data may prevent referees from, for example, following a frail of mathematical 
reasoning, replicating an experiment, or locating an archival document. 25 As the amount of 
published scholarly work expands, plagiarism and photo-manipulation detection software is 
becoming more commonplace. While some high-visibility journals are considering turning to a 
more aggressive form of peer review for controversial work — including giving referees access to 
underlying data — this is extremely expensive. 26 Fraud detection also raises questions about 
whether scientific debate should take place behind the closed doors of peer review or in an open 
forum following publication. 27 

• Research linked to specific stakeholders, for example, government or private industry agendas, 
may have overstated conclusions (e.g., Guterman 2005). 
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Information wastage 

• The publication system produces wastage in the form of worthy research that is never published 
and thoughtful reviews of work that are never addressed. 

Reforming the peer review model 

In light of the many criticisms of peer review given above, some individual scholars and groups of 
scholars have made slight, discipline-specific changes to the editorial and peer-review process in 
particular journal outlets. 28 Some scholars, frustrated with the limited journal publication outlets in their 
fields, have founded new online journals through grants and institutional or individual funding. Others 
have “tweaked” the editorial or peer-review process at existing society journals. Examples of these 
initiatives include: 

• New scholar-run journal publishers and digital press series that compete with high-impact 
commercial or university press publishers employing professional editors. Examples include 
Molecular Biology of the Cell , and the American Philological Association’s digital monograph 
series (in development). 

• Movements from a single-editor model to an editorial board model at journals grappling with a 
growing diversity of subfields. Examples include the American Political Science Review. 

• Experiments to encourage referees to review in a more detailed and timely manner. This can take 
the form of paying reviewers, instituting “social contracts” to encourage authors to undertake 
timely peer reviews, 29 or publishing the peer reviews. 30 

• Policies to reduce the burden on peer reviewers, including encouraging the re-use of peer reviews 
when a paper is rejected and resubmitted to a different journal, 31 or requiring authors to address 
the issues of one reviewer before sending their paper to a second. 32 

• Limiting the number of publications a scholar can submit for institutional review to help drive 
down the imperative to over-publish. This may help the problem of over-publication, but we are 
not aware of evidence that it does (cf. Waters 2004). 

In addition to these existing reforms, other scholars have suggested additional changes to the peer-review 
process, such as developing specific journal policies on the peer review of interdisciplinary work (Lee 
2006), and giving referees collaborator or coauthor status when extensive revision is necessary (Kumar 
2010). Furthermore, Jennings (2006) suggests that editors should engage in some reflective review 
themselves, occasionally looking at the impact of papers they have rejected in the past in order to keep 
their judgments in check. 

What are the costs of the publication-based peer review system? 

The costs of publishing peer-reviewed research are broken down and presented below as: publishing 
costs, peer review costs, the isolated costs of editorial work and managing peer review, and the costs of 
the total publication process. 

Publishing costs 

In the dominant publishing system, publishers pay the costs of first-copy publishing, which average 
around $1,700-2,000 per accepted article (Swan 2010). 33 These costs include the following (D.W. King 
2007, Houghton et al. 2009): 
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• Per-article publishing costs: manuscript review and processing, managing peer review, author 
communication, editorial input, managing illustrations/figures/ multimedia/data sets, verification 
of metadata, typesetting, proofreading, layout, and quality assurance of online material. 

• Per-issue publishing costs: composing and editing non-article content (e.g., table of contents, 
index, editorials, cover, review articles, news, and letters), composing and editing e-content, issue 
compilation, typesetting, layout. 

• Production costs: printing (including paper and binding costs) and/or online publication 
(including uploading to server and online hosting). 

• Dissemination costs : mailing or emailing, subscription management, and customer service. 

• Company support costs : appointing and managing editors and editorial board, managing 
reprints/off-prints/author fees, rights management, sales and marketing, maintenance of online 
systems, usage statistics, data conversion, managing journal lists, negotiations with societies and 
publishers, launching new journals and projects, and general overheads and management (e.g., 
administration, utilities, janitorial, financing and payroll). 

The costs of producing and disseminating material electronically have decreased for journals with large 
subscription bases (D.W. King 2007), but the human costs of managing the publication process have 
remained constant. Additionally, as publishers have lost their monopoly on scholarly dissemination in the 
digital world, more costs are expended on building information services around published work. 

Even though economies of scale dictate that many journals with high circulations (e.g., “top journals”) 
should have fewer costs per article or per subscription, top journals often have higher publishing costs due 
to high rejection rates and intensive editing (D.W. King 2007). (Journals do not generally receive 
recompense for the dead weight of rejected papers.) For example, the publishing cost per published article 
can exceed $3,000 in Science, compared to an average cost of $820 per article in Biomed Central 
journals. In contrast to the STM (Science, Technology, Mathematics) fields, journals in the humanities 
and social sciences have longer articles (19 pages on average), which can cost anywhere between $5,000- 
$10,000 to publish in print (or $2,500-$7,000 to publish online) (Waltham 2009). 

Who pays for publishing costs? 

In most cases publishers turn a profit by assuming ownership of the copyright of published work and 
selling access to protected content (Bergstrom 2001). Specifically, publishers pass their costs on to 
scholars and libraries in the form of book/journal purchase and subscription prices, 34 as well as additional 
page or layout charges to authors in some fields. 

Peer review costs 

Swan (2010: 27), using a very specific (and by no means generally accepted) cost model, estimated the 
costs paid by UK universities to scholars to support peer-review and editorial activities (which could be as 
much as 18 million GBP at highly-competitive research-intensive universities) often exceed the sum total 
of estimated costs that publishers pay to process articles as described above. 

Costs paid by universities in the form of faculty salary include: 

• $550 per article for each referee in researcher time, assuming that the average review of a journal 
article takes 4-6 hours (Tenopir and King 2000, Moms 2005, Mark Ware Consulting 2008). For 
one published article, then, the total peer review cost in researcher time could be $1,100 for two 
referees, or $1,650 for three. If an article is rejected and resubmitted to another outlet, then the 
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peer-review process generally begins anew, and another $1,100 or $1,650 is spent in referee 
research time (quickly totaling $2,000 or much more before an article is published). 35 

• Large opportunity costs to (the top) scholars, particularly in terms of time that could otherwise be 
spent on research, teaching, or publishing work. 

Who pays for peer review? 

Ultimately, scholars referee for free out of respect for the importance of the task, and to be part of the 
academic community (Brown 2009). Peer review is paid for by their university salary. Additional 
publisher-based incentives for reviewing can include small honoraria, receptions at conferences, discounts 
in author fees, free journal access, inclusion in a journal’s list of reviewers, and the prestige of being on 
the editorial team. 

The costs of managing peer review 

Presuming that the cost of managing peer review (not actually conducting it) can be isolated from the 
general publishing process, what would this activity cost? In a literature review on the topic, Rowland 
(2002) noted that journal costs to manage peer review can be on the order of $200-600 per published 
article, depending on whether editorial honoraria arc taken into consideration. These costs include: 

• Primary editorial review, or “triage,” to decide what gets peer reviewed. Other “editorial damage 
control strategies,” like submission fees, have hied to counter the deluge of submissions to 
journals that have a large market share (Abbott 2008). 

• Identifying and corresponding with peer reviewers. 

• Making editorial judgments and/or corresponding with the editorial team. 

As Bergstrom and McAfee (2005) describe, a faculty journal editor who handles around 100 papers 
annually would use about 20% of a secretary along with the associated space and materials; the sum total 
of which could be paid with a university overhead charge of about $12,000 per year. Although the online 
management of peer review has led to a decrease in mailing costs and increased efficiency for all parties, 
an online management system can still cost $1,000-$ 10,000 per year to maintain (Casler and Byron 
2009). 

Who pays to manage peer review? 

Although journals pay editorial staff for much of the logistical work, scholar-editors and editorial board 
members contribute a great deal of time and resources to many journals, often for little or no honoraria. 36 
Journals pay for the logistical work of managing peer review (and the intellectual work of professional 
editors), and are then reimbursed through subscription charges. Meanwhile universities pay for the 
intellectual work conducted by faculty editors and editorial board members through faculty salaries. 

Who pays the most for peer review? 

Although publishers may disagree, the largest inputs to the peer-review and publication process are made 
by referees, editors, and authors, and these are not usually priced in dollars. Rather, these arc largely costs 
that are mediated by reputation, respect, and social obligation, not money. 37 The university pays these 
costs, in that as a scholar’s reputation develops, so does his or her salary. Also, the university pays costs 
borne out by authors to prepare manuscripts and other non peer-reviewed journal content for publication 
and submission, including writing, obtaining any required permissions, choosing how and where to 
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publish, scif-archiving (if mandated), and editing in keeping with publisher format requirements 
(Houghton et al. 2009). 

The university budget pays for publishing scholarship by paying the salary of the scholar (and supporting 
facilities) to do the research, paying scholars to edit and review research, and again by purchasing 
published research from the publisher so that it can be read by faculty. 

Concluding thoughts 

In light of current economic factors and developments in publishing, the traditional form of publication- 
based peer review may be an overused, expensive system. Not only does the system tax the human labor 
expended (i.e., that of the referees), it may also slow the pace of science by both increasing the costs of 
“rejecting” the growing avalanche of scholarly publications submitted, and keeping ideas that are contrary 
to the current body of knowledge from emerging. 38 Seen in this way, the price paid for efficiency (in 
terms of readers’ time) may be very high. Further research into improving the peer-review process could 
address this peril by asking how and why papers should be reviewed, and whether different genres of 
papers and monographs might require different forms of peer review and dissemination than allowed for 
by the current model. The deluge of publications generated by scholars around the world, in response to 
institutional review requirements and government research assessment-type exercises, point to the lack of 
sustainability of the entire system. 

Working Paper 2 examines alternative forms of peer review and their possible utility in various fields. 



(2) NEW MODELS OF PEER REVIEW: REPOSITORIES, OPEN PEER REVIEW, AND 
POST PUBLICATION METRICS 



Introduction 



The importance of a piece of scholarly work can be weighed at various stages and in various ways. 
“Light” peer review occurs before a piece is formally published and ranges from the informal circulation 
of drafts among trusted colleagues, to giving papers at seminars and conferences, to posting work online. 
The premium placed on speed has enabled scholars in fields such as physics, math, bioinformatics, and 
economics to share “penultimate” drafts in discipline-specific repositories (as working papers or 
preprints) or on personal websites. Some traditional and newer publishers arc experimenting with open, 
volunteer-based reader-generated commentary on “pre-published” material; such approaches attempt to 
enable the academic community-at-large to decide publicly what is useful and what is not. 

Bibliometrics — particularly citation indices and the impact factor — provide scholars with a proxy to 
gauge the impact of their own work and filter formally published material post-publication. They can also 
inform institutional review and/or the allocation of research grants. A wider array of metrics is becoming 
available in the digital environment, and novel ways of assigning quality and impact to scholarly work are 
emerging. These include citation counts, bibliograms, webographies, social bookmarks, download metrics 
of articles, and quantitative analysis of reader-generated open commentary, ratings, and blog coverage. 39 

While our examination of new models of peer review is by no means exhaustive — moves toward peer 
review “reform” arc quite numerous — we suggest that the fundamental role of informal and formal peer 
review remains the same: to assess and improve the quality of scholarly work and act as a distillation 
mechanism. 
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Pre-publication peer review venues 
Repositories 

In addition to soliciting feedback on work-in-progress via informal networks, the posting of well- 
developed in-progress scholarship on personal websites or in institutional repositories (IR) or 
disciplinary-based repositories (such as arXiv, SSRN and RePEc) is common in some high paradigm 
fields with low commercial value, such as physics and economics. While the posting of working papers in 
a repository does not provide scholars with a “formal” peer review service (and therefore alone cannot 
indicate the impact of a piece of work on a field), a level of informal, light review often occurs . 40 

For example, users of the arXiv receive a “cursory” endorsement on preprints and can then solicit 
reactions and address errors though community-based informal review by providing an email address . 41 
This frequently acts as an intermediate or parallel step in the peer-review process when papers undergo 
the “expert” peer review provided at the journal level, although not all arXiv preprints are formally 
published elsewhere. Preprints that are formally published arc annotated with the updated bibliographic 
information. As such, arXiv is an example of how dissemination models prior to formal peer review can 
work well in some fields, but certainly not all (see below). More research is needed to understand how 
many arXiv preprints arc formally published elsewhere and how much “unnecessary” peer review may be 
avoided by operating in such a system. 

As we have showed in our recent research, many scholars throughout the physical, biological, and social 
sciences, as well as the humanities, are resistant to posting working papers in such a manner. They may 
be wary of sharing “unfinished” work for fear of being scooped or getting it “wrong” in early drafts, or 
early sharing may be outside the disciplinary tradition of a field. Molecular and cell biology, for instance, 
may be unlikely to move to an arXiv model because grant funding in the field is highly competitive, there 
is tremendous commercial potential, there is a fast turnaround time to publication (with multiple outlets 
available), and scholars already face an overload of information (which likely would be exacerbated 
under a working paper/preprint system). Indeed, this is probably the case for all fields that are fast- 
moving, well-funded, highly competitive, and have commercial potential; chemistry would be such a field 
as well. 

Conferences and seminars 

Although conferences and seminars in all fields function to get early ideas “out there,” enhance 
reputation, and widen networks, the degree to which conference papers are circulated and/or published 
can vary by discipline and by individual conference (or, indeed, by individual preference ). 42 While 
sharing drafts of work in both repositories and via conferences enables scholars to obtain a “light” review, 
it does not substitute for formal peer review in any way. One notable exception to this model is computer 
science, in which peer-reviewed conference papers are penultimate publications and are rigorously peer 
reviewed, indeed much more rigorously peer reviewed in terms of number of reviewers than most journal 
articles . 43 



DRAFT Working Papers 



Revised 4.22.10 



13 




3SHE/AWMF Workshop on Peer Review 



vpril 5-6, 2010 



“Open” peer review experiments by publishers 

Attempts to potentially extend and supplement conventional peer-review procedures through a more 
social and participative reader-generated approach have also occurred within the existing publication 
system. Peer commentary is offered by random readers or friends, rather than organized or selected for by 
editors. In contrast to the classic single- or double-blind peer-review process, which is typical of many, 
but not all, traditional models 44 in an open peer-review system, the written reviews are available to all 
readers of the work. 

Open peer review of journal articles 

• One of the most oft-cited examples of an online platform featuring post-publication peer review is 
PLoS ONE. The journal provides post-publication tools to indicate the quality and impact of a 
piece of work, and readership-based commentary on research articles. While submitted papers 
undergo a form of internal pre-publication peer review, all “technically sound” papers are 
published. (A scan of articles suggests that reader comments are, in fact, rare.) We suspect that 
the most competitive scholars will continue to submit their most important work to more 
prestigious, traditionally peer-reviewed outlets. 

• In Nature ’ s short-lived open peer review experiment, the journal gave authors the choice to have 
their submissions posted online to solicit commentary by readers alongside the conventional peer- 
review process. 45 The experiment ended after several months because editors found that there 
was a marked reluctance by readers to offer open comments, and those reviews that were offered 
were not more helpful than the conventional blind reviews (Greaves et al. 2006). We suggest that 
this response is not an aberration, but rather that such forms of open peer review are antithetical 
to the disciplinary cultures that Nature serves primarily. 

• The Electronic Transactions on Artificial Intelligence (ETAI), an electronic journal also 
implemented a (now ceased) commentary-based peer review. Papers were first posted online and 
then reviewed, with comments openly posted on the pages prior to the official review process. 

Open peer review of monographs 

Some recent research shows that in new open access options for book publishing, managing peer review 
continues to be crucial, particularly double-blind peer review, as the top indication of quality assessment 
(Adema and Rutten 2010). Nevertheless, experiments with a more open form of peer review have taken 
place: 

• Willinsky’s (2009) Open Monograph Press (OMP) is intended to support new opportunities in 
monographic publication, including a “new generation” of established presses, new players, and 
independent authors looking for increased publishing options and improved quality. It proposes 
an iterative peer-review model, which involves soliciting pre -publication peer reviews from a 
scholarly community that are posted online. Authors then have a chance to refine their work 
based on preliminary feedback. While it is too early to gauge its success, an OMP model strives 
to minimize publishing costs and provide open access with some form of peer review. 

• Digital book projects such as Gutenberg-e and If:Book have experimented with electronic 
monographs. 46 The latter has tested a model of blog-based open peer review, which can operate in 
parallel with formal university press review. In one such experiment involving a media studies 
monograph, the open comments, as well as the standard MIT Press reviewer comments, were 
both deemed useful by the author (Wardrip-Fruin 2009). 
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As such experiments proliferate, it will be important to assess what portion of “friends” contribute to such 
open comments in these venues, and whether the overall impact exceeds the normal levels of peer review 
that authors garner in sending drafts to trusted colleagues prepublication and that publishers garner in 
sending manuscripts to blind referees. 

Assessing more open publication-based peer review models 

There may be resistance to community-based peer review models for several reasons, which include: 

• Established publishers already have an exceptionally difficult time recruiting competent 
reviewers. Scholars’ limited time and conventions in the field are not likely to support volunteer- 
based commentary on non-peer-reviewed work posted online. 

• Some Pis actually ban young scholars in their labs for fear that they will say too much in their 
comments and risk being scooped (Harley et al. 2010). 

• There are currently no means to credit people who make important comments or contributions to 
the published work (Harley et al. 2010). 

• Some scholars express concern over writing comments online (and therefore reacting 
immediately /prematurely to research) because there is a risk of “getting it wrong.” 

Among readers, there is skepticism toward volunteer-initiated peer commentary: 

• Will readers trust findings that are openly peer reviewed by a readership, which in some cases is 
in lieu of expert peer review? Motivating the right peers to provide commentary can be 
problematic. 47 Should peer review be entrusted to those who have nothing more pressing to do 
with their time than to rummage through undistilled scholarly material (Harnad 2000)? 

• Scholars prefer polished and vetted work as a way to deal with information overload. 

Assessing formally published material post-publication 

There arc many ways in which scholarship receives post-publication review. Most traditionally, it takes 
place through the scholarly process itself whereby scholars build on other scholars’ work via publication. 
More recently, there are also forms of post-publication review that are driven by varying complex 
equations/algorithms that reflect consumption, engagement, and participation patterns. New review tools 
attempt to better understand how published work is used, and create filtering mechanisms so that “good” 
scholarship rises to the top and reaches its correct audience. 

Bibliometric data 

Bibliometrics refers to various methods used to quantitatively assess publication data using analyses of 
recorded information. Bibliometric methods can act as a proxy for the value of scholarly work and enable 
academics to measure and present the impact of their scholarship in new ways. 48 They can also act as a 
distillation tool for some scholars by identifying the highest impact outlets and the most frequently cited 
work. 

Citation analysis, or the study of the frequency and impact of scholarly citations, is one well-known 
bibliometric method. Citation counts can examine the impact of (a) a research article (i.e., the frequency 
with which an individual article is cited), (b) an author or multiple authors (i.e., the number of times an 
author is cited by others), or (c) a journal (i.e., impact factor, a measure of the citation frequency of papers 
in journals which is often used to compare journals across fields and is equated with journal stature). 49 
Using algorithmic analysis based on multi-dimensional factors, new software tools can mine, rank. 
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cluster, and analyze material producing a range of user-engagement and participation metrics. 50 While all 
metrics are quantitative by nature, some believe that bibliometric data in a digital environment can 
provide a “computable context” in the form of more nuanced and holistic information about scholarly 
publications (Jensen 2006). 51 Bibliometric techniques take many diverse forms, including: 

• Citation counts harvested from the research literature by subscription-based citation indexing 
services (such as Scopus or ISI) 52 or open access databases (such as PubMed Central). 53 Google 
Scholar also has citation functionality. 54 

• Co-citation analysis refers to the methods used to correlate citations and can reveal significant 
clustering between research papers, journals, and authors. Co-citation analyses are also useful in 
mapping the intellectual structure of a discipline (Garfield 1993). A bibliogram is one such co- 
citation tool that establishes links between clusters of scholarly material (including authors, 
journals, and subject headings) and can show who an author cites and who cites an author. It 
generates statistical trends of search results though text mining. Along similar lines, a 
webography lists websites (instead of books and journal articles) and provides information about 
online resource citations. 

• Using data mining techniques, patterns of online research article usage can be harvested. Web 
usage can include HTML pageviews, PDF downloads, and XML downloads, and can be 
harnessed for both in-progress work and final archival publication. 55 

• Readers can also “rate” a piece of work by assigning it a numerical quality rating. This is often 
performed in conjunction with open commenting as in the case of PLoS ONE. 

• Blog coverage can provide an alternative indicator of an article’s value, and article citations on 
blogs can be aggregated by tracking and indexing services for scholarly blogs like 
ResearchBlogging and Nature Blogs . 56 

• Social bookmarking data can indicate the value of a piece of work. Readers can use bookmarking 
providers such as CiteULike and Connotea to bookmark and share articles of interest. 

• Post-publication article expert recommendations alongside ratings help distill further the glut of 
scholarly work. In the sciences, for instance, the “authoritative” online service Faculty of 1000 
(F1000) involves select scholars highlighting and evaluating their “picks” for important research 
articles in a field. 57 

• Some emerging metrics depart from both the journal and the article as an indicator of value, 
counting awards, funding received, and other indicators of “scholarly productivity.” Examples 
include the h-index or Hirsch number (Hirsch 2007) 58 and the Faculty Scholarly Productivity 
Index. 59 



Problems associated with use of bibliometric data 

As Bollen et al. (2009) note, scientific impact is a multi-dimensional construct that cannot be measured 
adequately by any single indicator. 60 While historically bibliometric data provide readers with a 
“quantitative” measure in the form of a citation count, scholars need to “dive down” if they want to draw 
out “qualitative” value and contextual meaning. There are various criticisms made against efforts to 
quantify quality, including: 

• Reliance on bibliometric data in the institutional review of scholars can exacerbate the problem of 
outsourcing the judgment of scholarship from the institution to an outside proxy. 61 For example, 
the impact factor is widely misused; it was developed to measure the long-term relative strength 
of a journal, not to serve as a proxy measure for the importance or quality of individual journal 
articles included within the journal. Moreover, the impact factor is undermined by the publication 
of poor articles in good journals, something exacerbated by the inclusion of famous but unworthy 
authors, sloppy reviewers, and biased editors. 
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• A fundamental problem with metrics, which include the much maligned impact factor, is that they 
can be easily gamed (e.g., Haque and Ginsparg 2009; Borgman 2007). For instance, they can be 
abused by self-hits and inflated by automated web-crawlers (McKiernan 2004). 

• Article-level metrics are also insufficiently nuanced because they tend to focus on those with 
universal appeal or the most controversial papers (and thus reward negative citations, Akerman 
2006). 

• Rating services do not provide evaluations or reviews of articles. F1000, for instance, is cited as 
unhelpful because scholars merely “flag” articles rather than indicating the reasons for the 
flagging. 

• Do viewing and/or download statistics accurately reflect consumption practices? A scholar may 
navigate to or download a paper but not actually read it. 

• The coverage of much citation data is partial at best, generally including only top-ranked journals 
or specific disciplines (Van de Sompel and Lagoze 2009: 196). 

• Depending on the field, the impact of an article or author’s work can take years to be reflected in 
standard metrics, and therefore can be missed by most tools. 

The real problem with metrics is that they substitute quantitative measures (often of dubious or at best 
limited value, and which can be easily gamed) for informed and thoughtful judgments by competent and 
responsible peers. 

Book reviews 

Book reviews represent one area where post-publication peer review actually works. A book review 
su mm arizes and evaluates a long-length publication for academic communities who have not yet read the 
book. These reviews are often extremely detailed and play an important paid in a scholar’s institutional 
evaluation. They are considered both in terms of what the reviews say, as well as where they are 
published. (Book reviews in prominent outlets also garner credit and prestige for their authors, and are 
included on author C.V.s.) In the humanities and social sciences, online review forums such as the Bryn 
Mciwr Classical Review and FI-Net listservs are commonly used for book reviews. Some of these forums 
enable the discussion of academic publications via comments posted on existing reviews. 

Concluding thoughts: The staying power of traditional peer review 

While conversations about new models of peer review abound, conventional peer-review practices, often 
taking the form of single- or double-blind peer review performed by experts in the field, remain firmly in 
place. Moreover, attempts to extend formal peer-review procedures have generally failed because 
ultimately scholars trust established publishing outlets and appear to avoid informal reader-generated 
open commentary. 62 While new metrics based on user-engagement and participation are increasingly 
prevalent, at present they seem to supplement (rather than substitute for) classic peer review underpinned 
by conventional bibliometric data (Adler et al. 2008). The current system is, thus, generally cited as 
successful, waits and all, insofar as it “picks out the best and gets rid of the worst.” 63 

Given the widespread staying power of publication-based peer review across fields, Working Paper 3 
examines alternative funding models for providing access to peer-reviewed research output. 
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(3) OPEN ACCESS: GREEN OA, GOLD OA, AND UNIVERSITY RESOLUTIONS 



Introduction: Non-proprietary peer review 

The dominant publication-based peer review system for article-length work is based on a “proprietary” 
model of peer review; authors pay for services rendered by transferring their copyright to the publisher. 64 
Publishers then recoup their costs by charging readers (or their institutions) for access to peer-reviewed 
work. This often prevents large swaths of peer-reviewed research from being accessed by scholars and the 
general public (e.g., Bide and Wise 2010). 

In response to criticisms of proprietary publishing, some scholars, disciplines, funding bodies, and 
institutions have called for new publication models allowing free open access (OA) to peer-reviewed 
scholarly work. There are several ways in which authors can negotiate alternatives to the proprietary 
publishing model; these generally fall under the rubric of either Green OA (archiving versions of 
published work), or Gold OA (paying publishers a per-article fee in lieu of transferring copyright). In 
most OA models, authors retain copyright of their published work in order to re-use or circulate it as they 
see fit. 65 

Green OA 

Many proprietary publishers allow authors to use some version of published work for non-commercial, 
personal purposes, such as posting to a personal website, emailing to colleagues, or distributing to 
students (although this is highly variable by publisher). 66 Generally, however, Green OA refers to the 
deposit of work into a formal repository, including: institutional (e.g., eScholarship) , governmental 
(e.g., PubMed Central) , or disciplinary (e.g., arXiv , RePEc) . Because these repositories may act as a 
“back door” for readers to access published (or close-to published) material, author deposit may require a 
special agreement with the proprietary publisher (e.g., rewriting a copyright contract), and/or additional 
fees. Some examples of copyright negotiations include: 

• Granting a publisher a license to publish (rather than transferring copyright), which could include 
non-exclusive or limited rights. 

• Granting first rights or one-time rights to a publisher to allow authors to stipulate a length of time 
(“embargo period”) on the publisher’s holding of exclusive rights to a piece of work. 

• Retaining the rights to the preprint or the last unpublished version of a piece of work. 67 

• Purchasing reprint rights to published work. 

• Ignoring the publishing agreement. A 2008 survey of scholarly publishers found that journals are 
tightening their restrictions on the use of final published content, particularly in large, well- 
trafficked repositories (Cox and Cox 2008). Questions remain regarding how frequently scholars 
flaunt such policies and how rigorously they are policed by publishers. 

Gold OA 

Several flavors of Gold OA journals of varying quality have appeared in recent years, 68 funded by a 
variety of business models (Crow 2009). The author-pays model, for example, charges publication fees to 
an author. Although there are some perceptions that author-pays constitutes a form of vanity publishing 
(Harley et al. 2007), leads publishers to “over-publish” work for profit (Esposito 2004), or otherwise 
threatens traditional peer review, peer review remains compatible with and important to quality OA 
publishing (Harnad 2000, APA/AIA Task Force 2007). Indeed, the creation of prestigious OA journals is 
establishing a niche for OA publishing in some disciplines, such as the Public Library of Science (PLoS) 
in biology. In another funding model, some journal publishers offer the option of granting immediate OA 
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to individual articles by charging additional author fees. This creates a hybrid “open-access, traditional- 
subscription-based model.” In Gold OA, authors retain copyright of their published work: 

• Creative Commons licenses provide a range of options to authors to govern the use and citation of 
their work. This is often the license of choice for born-OA journals, such as those published by 
PLoS. 

• The Springer Open Choice program allows authors to avoid the exclusive transfer of copyright to 
the publisher, as well as ensure that published work is published OA online, for a fee. (As noted 
below, such arrangements are not without detractors.) 

• Some institutions will underwrite publication fees in OA outlets, as discussed in detail below. 69 

Creating open access to scholarly work 

Despite the benefits offered by Green and Gold OA publishing models, conventional publishing outlets 
continue to be the preferred choice for faculty in the U.S. In 2007, small studies demonstrated that many 
faculty are resistant to or apathetic about self-archiving (Hansen et al. 2007), and few consider or manage 
copyright in their publishing choices (The University of California Office of Scholarly Communication 
et aZ. 2007). Compounding this problem, traditional journals remain largely unenthusiastic about 
reforming the proprietary publication model to allow immediate OA (Shavell 2009). Author-pays models 
may be far less sustainable in the humanities and social sciences than in the sciences (Waltham 2009). As 
described below, academic research funding bodies, scholarly societies, and universities have responded 
with various policies to encourage faculty to exercise their copyright and create OA to versions of their 
published work. The hope is that such resolutions will not only increase public access, but also have a 
dampening effect on extreme journal prices. 70 

Government and academic research funder policies 

There has been gradual and accelerating support for OA policies among funding bodies in the U.S. and 
other countries. 71 Debate over how more public access can be created to federally funded research in the 
U.S. is fierce (Campbell et al. 2010), 72 as evidenced by the Office of Science and Technology Policy’s 
recent online forum on the topic. A key note of opposition is how publishers can be compensated for the 
value-added they provide in the editorial and publishing process (Mabe 2010, Smith 2010). Although 
federal agencies are unlikely to make new forms of publication mandatory, due to pushback from the 
publishing profession and some scholarly societies, the threat of encroachment on scholars’ external 
funding could nevertheless be an important driver of OA in some form. Currently, funder policies take 
Green and Gold forms: 

• In 2007, Congress mandated that all investigators funded by the National Institutes of Health 
(NIH) must deposit their accepted, peer-reviewed manuscripts into PubMed Central within a year 
following publication (Green OA). 73 Authors deposit work themselves, or journal publishers may 
deposit articles automatically. 74 Some have suggested that the year embargo period is too long. 

• In 2007, the Howard Hughes Medical Institute (HHMI) and Elsevier reached an agreement 
wherein Elsevier deposits author manuscripts of articles into PubMed Central (Green OA) six 
months following publication, and HHMI pays Elsevier an additional fee for the service 
directly. 75 

• Additionally, some funding bodies (e.g., the NIH and UK’s Wellcome Trust) may pay publication 
fees for grantees to publish in a Gold OA outlet. In some cases, these policies combine Green and 
Gold OA. For example, authors funded by the Wellcome Trust and publishing in Cell Press 
journals can pay a per-article fee of $5,000 for the journal to deposit the final manuscript into 
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PubMed Central upon publication and publish it online with immediate open access. The 
Wellcome Trust then reimburses authors. 

Scholarly society policies 

The revenues from journal subscription support the activities of many scholarly societies, and some very 
powerful societies can be expected to use legal actions and political pressure to stop appropriation of the 
publishing function by other entities (C.J. King 2007, Waltham 2009). It is important to note, however, 
that not all societies contribute to out-of-control subscription costs. And some societies are instituting 
various forms of OA access to their journal publications (following embargo periods to protect 
subscription revenue). 

Society policies on Green OA 

While some society journals ensure OA to articles on journal websites and in repositories like PubMed 
Central following an embargo, 76 others give authors the personal responsibility to deposit a preprint 
accepted for publication. 77 The degree to which other societies simply look the other way is not known. 

Society policies on Gold OA 

In a more innovative move, the Sponsoring Consortium for Open Access Publishing in Particle Physics 
(SCOAP 3 ) is attempting to change the six flagship journal publications in the field from a subscription to 
an OA model (The SCOAP 3 Working Party 2007). Rather than charging authors directly for publication 
fees, SCOAP 3 would act as a financial mediator between libraries and publishers by pooling donations 
from member libraries (which are diverted from subscription payments) and dispensing them to journal 
publishers to support Gold OA publication. 

University resolutions 

Various universities, including elite institutions, have paved the way for reserving a bundle of non- 
exclusive rights to the university. 78 It is arguably premature to predict how effective these policies will 
be. It is clear, however, that they have set a standard that will likely make it easier for other institutions to 
follow with similar policies. As with funding bodies and scholarly societies, university resolutions have 
largely taken two (nonexclusive) forms along the lines of Green and Gold OA. 79 

Institutional resolutions for Green OA 

Various institutions have had policies in place to encourage faculty to deposit articles in their open-access 
IRs for several years. 80 In 2009, several universities began instituting stronger resolutions regarding 
faculty deposit (Suber 2010). Important elements to note are the voluntary nature of these resolutions are 
the opt-in and opt-out features. The resolutions at Harvard University and the Massachusetts Institute of 
Technology are summarized below. Both have an opt-out option for faculty. 

Harvard University’s school-based policy states: “Each Faculty member grants to the 
President and Fellows of Harvard College permission to make available his or her 
scholarly articles and to exercise the copyright in those articles; provided that the articles 
are not sold for a profit.” 81 Faculty members then provide an electronic copy of the final 
version of the article, which is then distributed through Digital Access to Scholarship 
(DASH) , a new, university-wide IR. 
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Similarly, Massachusetts Institute of Technology’s (MIT) university-wide policy states: 

“Each Faculty member grants to the Massachusetts Institute of Technology nonexclusive 
permission to make available his or her scholarly articles and to exercise the copyright in 
those articles for the purpose of open dissemination.” 82 The Provost’s Office then makes 
the scholarly article available to the public in its existing IR (DSpace) . 

Other university resolutions in the U.S. include: Stanford University’s School of Education, Boston 
University’s university-wide OA model) , and the University of Kansas’ university-wide OA model . Many 
of these resolutions have similar features, including: 

• Relevance to article-based work only (that is royalty-free). 

• A not-for-profit stipulation on the university’s use of copyrighted materials. 

• A focus on the author deposit of final manuscripts of articles (post-review but pre-publication). 

• Requirements for faculty to pursue non-exclusive licensing agreements with publishers through 
the use of an addendum to publishing contracts. 83 Scholars may retain copyright themselves. 

• An allowance for authors, not publishers, to opt-out of the requirement using a waiver. This is 
especially important for younger scholars who lack negotiating power with prestigious publishers. 

• Institutional representatives responsible for interpreting the policy and resolving any disputes. 

Questions and criticisms of passed resolutions 

Although these resolutions do enable open access to a peer-reviewed form of published articles and 
support a faculty’s diverse publishing behaviors, many outstanding questions remain. 

• What is the status of published forms of articles to which universities would have this license? 
Could they assert the right to use the final published form of an article per a nonexclusive 
license? If not, how could the bibliographic integrity of different forms of an article be preserved? 

• Both MIT and Harvard have been criticized for their vague opt-out policy, which enables scholars 
to appeal the policy on a case-by-case basis. Researchers could technically acquire a waiver to 
maintain exclusive rights, allowing them the freedom to proceed as they wish with publication. 

• At MIT, Suber (2010) notes that the new policy does not specify the method of deposit. Faculty 
are merely to make their work available for deposit, and the Provost’s Office will somehow 
ensure that articles make it into the repository. On the other hand, requiring faculty to do the least 
amount of work possible is perhaps a positive way to ensure faculty support and compliance. 

• Coordinating publication contracts for collaborators across institutions can and will present 
exceptional challenges. 

• Combined with funder resolutions, these university resolutions create parallel universes of 
repositories. How can deposits to multiple repositories be coordinated? 84 Rather than providing 
alternative versions of published articles free of charge locally, should IRs simply provide links to 
full-text publications? Moreover, should IR contents be grouped for more efficient management 
and sustainability? Is there a need for a federated single search? 

• There is variation in the research and publication needs across disciplines. For example, science 
researchers oppose embargoes more than scholars drawn from the social sciences and humanities, 
due to the need for more timely access to research output (Fry et al. 2009). 

Contested and rejected OA resolutions and some reasons for faculty opposition 

OA movements at other universities have been rejected or watered down by faculty. Their outcomes may 
be explained, in part, by the differing approaches and emphases at each institution. 
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University of California (UC) (from The UC Office of Scholarly Communication 2007) 

• Despite the fact that the proposal had been under discussion in Senate forums for over a year, the 
vast majority of faculty was unaware of the proposal. 

• Faculty questioned university competence in the publishing arena, and why the university should 
take precedence over discipline-specific bodies and government agencies. 

• There were concerns that “forcing change on publishers” might adversely affect the quality and 
sustainability of the publishing system, as well as scholars’ own publishing relationships. 

• Faculty were concerned that individual authors would face a higher cost burden. 

University of Maryland (UM) (from Davis 2009a, Flackman 2009, Suber 2010) 

• There were overall low levels of faculty awareness of open access issues. 

• The UM resolution combined Green OA archiving with Gold OA publication, as well as other 
issues. It, therefore, lacked clarity. Faculty mistook the discussion of Green OA for a mandate of 
Gold OA; consequently, faculty believed it impinged on their academic freedom to decide where 
to publish. 

• There was no opt-out clause (as at Flarvard) to enable scholars to acquire permission to submit 
their work to journals regardless of access policy. 

• The discussion in favor of the resolution was based largely on an economic argument, linked to 
journal pricing, which was not bolstered by har'd data. A moral or other argument may have been 
more compelling? 

University of Virginia (UVA) (from Mullafiroze 2009, Park 2010) 

• The implementation of the repository was not clear, and several faculty called for discipline- 
specific approaches. 

• Faculty from the departments of Physics and Mathematics were concerned about coordinating 
publication contracts with collaborators at other institutions. 

• Faculty from the School of Architecture and Department of Art felt that they would be forced to 
opt-out of the text-only repository, due to the highly visual nature of their work. 

• After a lengthy debate in 2009 over a proposed Green OA resolution, faculty were still concerned 
about the mandatory nature of the proposed resolution. The UVA Faculty Senate’s Task Force on 
Scholarly Publications and Authors’ Rights then changed the language to allow faculty to “opt in” 
if they wanted to contribute to the repository. This revised “voluntary” resolution was passed in 
20 10. 85 

Institutional resolutions to underwrite existing publication charges — Gold OA 

In addition to, or in lieu of, an open access resolution, some universities are committed to underwriting 
“reasonable” publication charges to support OA publication. 86 The short-term goal is to enable all faculty 
to publish in Gold OA outlets. This goal can be met by providing a pool of money in the institution or 
library for faculty publication, or by universities making general agreements with particular - publishers. 87 
The long-term goal is to restructure publishing costs by helping publishers transition from subscription- 
based to author-pays revenue models (in which the university pays author fees on behalf of its faculty). 

Questions and criticisms of this policy 

• University interventions in the form of fees will do nothing to address the high price of 
commercial journals, but rather simply provide another revenue stream for the publishers 
(e.g., Kaemper 2009a, 2009b, Davis 2009b, Poynder 2009). In fact, it may be that smaller 
journals with low circulations are the better candidates for author-pays models (D.W. King 2007). 
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• Will authors with grant funding have to pay their own publication fees, while other authors are 
subsidized by the university? If so, will a pool of money to support publication be part of a 
startup package for junior faculty in some fields? 

• Major internal shifts in university budgeting are complicated and challenging. How can 
institutions pay for the elevated costs for the period of transition from the old to the new system 
(which likely calls for the temporary, simultaneous support of both) (King 2005, Shieber 2009)? 

Concluding thoughts: How can a sufficient group of committed faculty be mobilized? 

In order to successfully pass a university resolution, it appears necessary to include considerable scholar- 
to-scholar discussion to persuade faculty to (1) agree to a university licensing agreement, and (2) publish 
in the new venue. In deciding what this will take, several considerations could prove helpful: 

• Faculty must be educated on issues of OA and copyright, 88 with their discipline-specific 
publishing practices taken into consideration (Fry et al. 2009). Talk concerning the requirement 
to self-archive refereed journal articles (a Green OA resolution) must be disentangled from advice 
concerning whether or not to publish in Gold OA journals. 

• Universities should make clear that new requirements do not run counter to tenure and promotion 
requirements. The Provost’s office could play a key role vis a vis external reviewers and granting 
agencies by explaining that these are legitimate forms of scholarly communication. 

• Support must be built from the grassroots level in individual departments, rather than starting the 
conversation with the University Senate, for example. The resolutions at Harvard and MIT were 
spearheaded by faculty scholars, including Stuart Shieber, Robert Darnton, and Hal Abelson, 
(Hackman, 2009). Deans and chairs play crucial roles and have access to local resources. 

• The input of the university librarian must be secured to ensure that the library can support 
alternative services relative to the storage of journal articles. 

• Different considerations may need to be made for publishers of articles in the sciences, versus 
those in the humanities and social sciences (Waltham 2009). 

• A resolution (or even a mandate) does not automatically ensure author deposit (as witnessed by 
the 60% author deposit rate to PubMed Central following the NIH mandate). 89 Repositories must 
feature easy-to-use tools for scholars (or publishers) to conveniently deposit work in a timely 
manner. 

• There are several external initiatives that provide a roadmap of best practices, including: the 
SPARC Campus Open Access Policies project , and Enabling Open Scholarship (EOS). In the 
UK, JISC has assembled InfoKits on repositories, resolutions, and advocacy literature. 
Additionally, in 2006, the AAU, ARL, CNI, NASULGC, and SPARC cosponsored a forum on 
“ Improving Access to Publicly Funded Research: Policy Issues and Practical Strategies .” 

The following arguments (some of which were advanced by participants at the April 2010 meeting) could 
also prove helpful in mobilizing faculty: 

• The proprietary publishing system removes scholars’ rights to the fruits of their labor. Scholars 
must secure permission to re-use their own data, figures, and content published in proprietary 
journals. The publication system would be more efficient if controlled by scholars and their 
representative communities. 

• If the university takes a moral stand (e.g., access to knowledge is a public good), it is beneficial to 
the reputation of the university. 90 The university has a right to lay claim to its faculty members’ 
output (The University of California Office of Scholarly Communication et al. 2007). In an IR 
model, the university would also be able to better measure and track the intellectual output of the 
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university, something which could be important in attracting additional funding, other research 
support, and talented individuals. 

• Beyond the academy, there arc large numbers of citizens (members of “disease” communities, 
inventors, struggling high tech start-ups, farmers, small business people, etc.) who want to read 
the academy’s research products but have great difficulty accessing them. 91 Per the “public 
goods” argument, it is precisely these individuals who could help the academy gain the political 
support needed to gain increased federal and state appropriations for research universities. As 
federal funding agencies and university faculty bodies consider deposit mandates, the political 
advantage that increased access to scholarship by the public brings should be a heavily weighted 
consideration. 

• The university has the clout to force change on publishers relieving individual scholars of the 
pressure (The University of California Office of Scholarly Communication et al. 2007). 

• The sum of money saved by moving to a Green OA or Gold OA publishing model could be 
diverted to support faculty research and teaching in other respects. 92 

Finally, some have argued that faculty need more high-quality OA outlets available to them in order for 
university-based resolutions to succeed (cf. Waaijers 2009). The creation of such outlets, faculty 
allegiances to existing stakeholders (societies and publishers), values regarding IRs, and concerns about 
university competence in the publishing arena arc discussed further in Working Paper 4. 



(4) CREATING NEW PUBLISHING AND PEER REVIEW MODELS: SCHOLARLY 
SOCIETIES, PRESSES, LIBRARIES, COMMERCIAL PUBLISHERS, AND OTHER 
STAKEHOLDERS 



Introduction: Building new outlets for scholarly communication 

Would the development of a wider array of acceptable publication alternatives address current and 
emerging scholarly communication needs as well as maximize the purchasing power of cash-strapped 
libraries? 93 What might possible dissemination models that attempt to wrest control of the process from 
for-profit publishers look like, and how might scholarly societies, university presses, libraries, 
commercial publishers, and other entities figure in such a reconfiguration? In examining new publication 
possibilities, it is important to consider what might work, what clearly does not, and how new forms 
might compete and/or coexist with current publishing models. Proposed alternative publication systems 
must incorporate the successful elements of peer review, as well as address the “glut” of information in 
the online environment, by providing effective filters to direct scholars to the most important and relevant 
information. 94 

As discussed in Working Paper 1, professional publishers provide a bundle of services, including: 
packaging new scholarly material into an existing or new brand, management of submissions and peer 
review, editorial oversight, copyediting/typesetting, layout, metadata, publication/dissemination, and 
publicity/marketing. Potentially, the publishing process could be made more efficient by unbundling 
particular' services. In order to discern who should be taking on what services in a new publication model, 
the discussion that follows will examine the current roles of each of these players and hypothesize 
possible alternative roles in a new publication system. (This discussion of roles is by no means 
exhaustive, but presents some of the most obvious tasks, services, and functions.) Among the central 
questions is whether peer review can be separated from publishing, and if so, who will do it and how will 
it be coordinated, if not by the publisher? Additionally, how may new on- and off-campus collaborations 
harness the existing publishing capabilities of new or established institutional players (Adema 2010)? 
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The role of scholarly societies 

Scholarly societies have traditionally been major players in the publication and peer-review process. They 
publish “flagship” publications, manage peer review and editorial work, organize conferences, maintain 
resource portals, and operate as lobbyists on behalf of their members. 

Functions/services scholarly societies could provide 

Although some societies (certainly not all or even the majority) face criticism over “too high” 
subscription fees and the outsourcing of their journals to commercial publishers (cf. Glenn 2008), they 
represent the natural community of peer referees. Journal editors are often chosen through society 
nomination processes, and these editors then appeal to specific society members to referee work 
submitted for journal publication. Societies are thus well-positioned to manage several editorial and peer 
review functions, including: 

• The intellectual work required to identify, plan, and launch new (OA) publication venues based 
on scholarly needs in a discipline. 95 

• The organization of peer review bodies to certify work that cannot get published due to financial 
restrictions at presses (Shulenburger 2007), as well as the lack of publishing outlets for digital 
scholarly products (Bates et al. 2006). 96 

• The nomination and reward of scholar-editors of publications. 

• Editorial and referee judgments on work published in repositories. 

• The creation of overlay journals, minimalist journals that provide peer review but not a 
publishing platform (Suber 2001). Still fairly speculative at present, an overlay journal would 
mine self-archived “raw” author manuscripts from repositories and carry out publishing functions 
such as peer review management, editing, and perhaps branding (Swan 2010). The actual 
published content would continue to reside in the repository, perhaps with an updated postprint 
reflecting any revisions and updated metadata reflecting the journal/society brand that carried out 
the peer review. The overlay journal would then link to the content via a Table of Contents. 

• The creation of other outlets to aggregate and filter published content. For example, virtual 
journals published by the Joint Institute for Nuclear Astrophysics aggregate articles from other 
publication outlets for easy consumption (Cybert et al. 2010). Likewise, society-specific RSS 
feed aggregators could bring various types of publications together for scholars (similar to 
functions provided by JJ-Net listservs in some humanities and social sciences). 

Obstacles faced by scholarly societies 

Given scholars’ trust in the current system of peer review, it seems that a disciplinary society may be 
better posed than a centralized entity, say a “National Institute of Peer Review,” to oversee and manage 
peer review dislocated from publication. But, scholarly societies depend on revenue from their 
publications to fund other important activities, and so they may need other stakeholders to support the 
costs and activities of publishing and preserving new publication venues if subscriptions are not 
continued at sustainable levels. Moreover, scholars working in interdisciplinary, emerging, and fringe 
areas may not find a single society representing their publishing interests. 

The role of libraries and repositories 

University libraries and librarians are experts in the access, organization, and preservation of scholarly 
material, and many have become directly involved in creating and managing “repositories” of otherwise 
unpublished work. These repositories can hold research specific to an institution (for faculty at an 
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individual university), research in a particular discipline (such as the arXiv, housed at Cornell), or 
research funded by a particular government entity (such as PubMed Central, hosted by the U.S. National 
Library of Medicine). 97 Repositories can be open access, or limited by subscription to a particular 
academic community. 

Lynch (2003) defines a university-based institutional repository (IR) as, “a set of services that a university 
offers to the members of its community for the management and dissemination of digital materials created 
by the institution and its community members.” Generally managed as an arm of the library, institutional 
repositories preserve and deliver open access to varied forms of their faculty’s work, including grey 
literature, educational resources, large amounts of graduate student work (Van Westrienen & Lynch 2005, 
Schmitz 2008, AAU 2009), and some peer-reviewed published material (or author preprints). They are 
also quickly becoming a crucial element of university resolutions for Green OA publishing. 

As noted earlier, disciplinary-based repositories may be preferred by most scholars because they target a 
specific and natural research community (Fry et al. 2009; Davis and Connolly 2007). Despite the number 
of different models and sponsors for repositories across the academic community, there are a variety of 
common functions repositories can and do serve. 

Functions/services repositories could provide 

Given the digital infrastructure and management expertise that repositories currently represent, could 
individual or collaborating repositories also form the basis of a new digital publishing model (cf. Crow 
2002, Davis and Connolly 2007, Markey et al. 2007, Shulenburger 2007, Flahn 2008, Smith 2008)? Some 
possible publication roles for repositories include: 

• Acting as an initial point of formal submission for publication. Depositing work in a repository 
provides timeliness and registration by providing a date stamp, which allows a scholar to lay 
claim to an idea (Davis and Connolly 2007). “Publishers” could then “harvest” publications for 
formal publication. 98 

• Acting as a giant publication venue. Submitted work could live in the repository and be submitted 
to a “stand-alone peer review service,” the results of which could be recorded in a public registry 
(Akerman, 2006). 99 The repository contents would become the final, archival publication. 

• Providing a publication platform for scholarly societies interested in increasing the distribution of 
less profitable scholarly work. 100 

• Providing a locus for digital or electronic publications produced by a university press. 101 

• Assuming publishing responsibilities (such as workflow management, platform hosting, 
metadata, and archiving) for smaller journals, start-up journals, those not yet available in library- 
accessible digital form, and those which are not available in journal aggregators and are therefore 
at a greater risk for cancellation (Ivins and Luther 2009). 

• Supporting formal publications by providing access at the back-end to large data sets, grey 
material, multimedia content, and other supplementary materials not typically housed in 
traditional publications. 

• Creating metadata, which is an important component of repository infrastructure since users (and 
search engines) want scholarship to be tagged (by title, creator, subject, description, publisher, 
rights, peer-review status, etc). 102 

• Providing download statistics and other quantifiable measures of the dissemination of an author’ s 
work (Fry et al. 2009). 
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Obstacles faced by repositories 

As some have indicated, the missions of libraries and publishers are not always compatible. 10 ' In 
particular, libraries and repositories are not well designed to do the following: 

• Decide how work should be published or packaged into journals or special issues (Brown et al. 
2007). Repositories can provide deep search tools and opportunities for scholars to be alerted to 
new deposits in particular areas of the repository, but they do not generally provide editorial 
oversight. 

• Handle the various genres of scholarship that exist across disciplines. The costs for developing 
platforms to host and maintain multimedia material are high, and few repositories guarantee file 
formats, with the exception of PDFs, over time (Smith 2008). 

• Use surplus funds to finance new publication models. Repositories require substantial financial, 
personnel, and technical investment (Markey et al. 2007). Managing the peer review and 
publication processes takes experience, time, money, and faculty input. If this process is 
transferred to the university, local faculty may bear an increased time burden as their services are 
called upon to support and inform new publication models. 

• Convince authors to submit a critical mass of material to establish robust, particularly discipline- 
based, repositories. 

• Exist in an integrative way with IRs at other institutions. An alternative model might see groups 
of institutions joining together in common, shared IRs. 

Disincentives for author deposit in IRs 

While the culture and values in some fields encourage authors to submit in-progress work to disciplinary 
repositories, institutional repositories frequently languish because faculty contribution is low (cf. Foster 
and Gibbons, 2005, Davis and Connolly 2007, McDowell, 2007, Marshall 2008, Wittenberg in Harley 
2008). The following are some reasons why faculty may be disinclined to support a new library or IR- 
based publication model. 

• Scholars’ lack the knowledge, time, and motivation to deposit their work in new outlets, 
particularly as IR user interfaces are perceived as unsatisfactory (Smith 2008). 104 

• Deposit in different types of repositories varies by field, with availability and speed to publication 
being the two largest motives for scholars to deposit (Fry et al. 2009). 

• The “brand” of a journal or publisher is important to scholars, both for prestige (in the case of 
authors) and quality control (in the case of readers). Many scholars see IRs as competing directly 
with existing publication options/subscription journals (although this may be less common in the 
social sciences and humanities) (Fry et al. 2009). 

• Some scholars may not want their peer-reviewed work to be disseminated alongside non-peer- 
reviewed material (though this is dependent on field). It may be that scholars use IRs to 
disseminate non-peer-reviewed work and withhold work due to be submitted to a journal (Guess 
2008a). 

• Publishers may have copyright restrictions that prevent scholars from depositing their work 
elsewhere, including in repositories. 

The role of university presses 

University presses in many ways represent the “publication strategy of the academy at large,” because 
they specialize in particular disciplines and, subsequently, their publication lists include authors drawn 
from outside the home institution (Shulenburger 2007). University presses provide expert guidance to 
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authors as they craft and edit their manuscripts for publication. In consultation with scholarly societies 
and senior scholars, university presses are can help shape scholarship in many fields, particularly in the 
humanities and social sciences. What kind of publication models could be created either by university 
presses alone or by partnership among university presses, 105 scholarly societies, the university library, and 
academic IT? 

Functions/services university presses could provide 

University presses currently provide a variety of expert publication functions: 

• Editorial oversight, including manuscript submission, copy editing, layout, and marketing. 

• Management of peer review or mediation between scholarly societies and library publishing 
systems. 

• Creation of specific publication genres and templates for the digital publication of both journals 
and monographs (cf. Willinsky 2009). 106 

• Linking the publication of certain peer-reviewed journals to disciplinary repositories or perhaps 
“bidding” for the rights to publish author manuscripts deposited in various IRs? 

• Creation and maintenance of research portals that aggregate repository-based peer-reviewed, non- 
peer-reviewed, and educational materials in a particular research area. 107 

• Development of open-access publication tools for faculty scholars interested in creating new 
journals, including submission and editorial management services (Ivins and Luther 2009). 108 

• Publicity and dissemination of scholarly materials and resources to relevant audiences, including 
at conferences, via listservs, etc. 

• Development of a new publishing model for specialized monographs based on print-on-demand 
technology and short-run technology. As Rose (in Harley 2008, p. 11) suggests, faculty 
associated with research centers at universities, for example, could develop a specialized 
monograph series with the quality secured by an editorial board and peer evaluation 
accomplished in much the same fashion as it is done at a university press. Publications would 
bear the joint imprint of the research center and a university press and both would accrue prestige. 

• Print-on-demand (POD) functions for work published in electronic form. 

Obstacles faced by university presses 

Despite their traditional expertise in the publishing arena, there are several limitations faced by university 
presses in the digital world: 

• University presses are not automatically able to handle the demands of new genres of publishing, 
being deeply entrenched in conventional publishing models. This includes having the 
infrastructure to create and post multimedia products, work that requires revision or versioning, 
back-end data, etc. 

• In order to make university presses an important paid of reforming research distribution strategies, 
they will have to be convinced to adopt the open-access outlook that librarians embrace 
(Shulenburger 2007). Yet an OA model presents problems for funding the significant editorial 
work presses provide. 

• University presses are generally associated with disciplinary specialties, which are not necessarily 
shared by their own institutions. As Shulenburger (2007) observes, university presses (and 
institutions more generally) risk losing their scholarly prestige if they are perceived as publishing 
their own faculty’s work de facto (such as that deposited in IRs). This complication could be 
mitigated by having editorial judgments made by individuals outside of the advancement review 
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process, or by moving to multi-institutional peer-reviewed repositories or collaborative press 
initiatives (King 2005). 

Concluding thoughts 

Although a future in which various stakeholders partner to create new publication models is fraught with 
logistical difficulties (Brown et al. 2007), there are many forms of publishing expertise distributed across 
the academic landscape. The key to moving forward with any new model is to reduce the enormous 
duplication among the services that different stakeholders can provide, 109 and focus on which services 
each can do best. The outstanding questions for launching successful collaborations between scholarly 
societies, repository venues, and university press activities — and there are several — may include the 
following: 

• In an OA publication model, who will pay for remaining costs of publication (which arc 
significant and include the salaries of copyeditors and content managers, as well as the “bits and 
bytes” of digital infrastructure and preservation)? How will costs be allocated across the 
university or between universities, particularly in the absence of reform to the current journal 
subscription/publication model? 110 

• If a publication is processed by a university press, published in a repository, and peer reviewed by 
an editorial board in a scholarly society, whose imprint will the final publication bear (that of the 
society, press, or library)? 

• How can new publication models earn legitimacy among faculty, and/or how can a critical mass 
of scholar-authors be mobilized to publish in alternate venues? How can new publications reach 
target audiences and wide dissemination levels, in competition with the extensive marketing 
campaigns maintained by commercial publishers? 

• How can new publication models be integrated into existing avenues for scholars to keep up-to- 
date with the literature? What new Internet or search tools are needed (cf. Suber 2008, 2010)? 

• What is the right balance between organizing publication by disciplinary theme, versus by 
institution? 

• And finally, what is the likely role of or response by commercial publishers to new publication 
models? Will they respond by creating new (for-profit) tools for mining, ranking, and analyzing 
new publication models (such as Elsevier’s development of Scopus)? Moreover, if scholarly work 
is increasingly available in OA form, wherever it is housed, will scholars simply turn to Google 
scai'chcs or new aggregation tools rather than new publication outlets to locate scholarly work? 
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ENDNOTES 



1 The March 2009 meeting was inspired not only by what was emerging from our research findings, but also by 
informal conversations with Donald Waters and Paul Courant. Also Courant’s blog post: Courant, Paul. 2008. On 
the Meaning and Importance of Peer Review. Au Courant. October 12. http://paulcourant.net/2008/10/12/on-the- 
meaning-and-importance-of-peer-review/ . 

2 The process and substance of peer review differs by field. Some professional schools are creating their own 
specialized criteria forjudging scholarly output, such as journalism, architecture, law, and environmental design. 

3 Sharing unpublished work openly is less common in competitive, time-sensitive fields (e.g., molecular biology). 

4 See, for instance, the APA/AIA Task Force on Electronic Publications (2007) and the peer-review of 
ethnomusicological websites in the Yearbook for Traditional Music . 

5 For instance, the impact of a scholar’s peer-reviewed publications is integral when reviewing a scholar’s grant 
application or tenure package, and the informal assessment that work-in-progress receives can influence where it is 
published (e.g., journal editors may approach scholars at conferences and invite them to publish in their journals). 

6 Although conference presentations, working papers, (some) edited volumes, blogs, and other non peer-reviewed 
work can help scholars to establish precedence for their work and may influence the evaluations written by external 
reviewers, they do not substitute for peer-reviewed publications in the institutional review process. (Exceptions to 
this include fields like computer science, where conference papers constitute penultimate publications.)This may be 
because, as Borgman (2007) observes, it is easier for institutions to measure a scholar’s outputs (in the form of 
publications), than to measure their inputs (e.g., in the form of research time and other activities). 

7 In so-called high-paradigm disciplines, such as economics or biology, there may be sufficient consensus across 
subfields to reward good work that is not published in the top general-interest journals. 

s At the root of institutional review is a fallacy that, “I’m an excellent researcher, the system has obviously realized 
that I’m excellent, and therefore the system must be excellent and there’s nothing wrong with peer-review” [NJ, peer 
review meeting, 03.05.2009], 

9 These two studies sampled scholars from the Thompson Scientific author database and ISA author database, 
respectively, with response rates of 7-10%. 

10 Archival journal publication has become a more formal affair, involving lengthy literature reviews and the 
polishing of form and style. This “finalizes” the research in a form that everyone in the field can accept. 

1 1 In particular, reviewers help authors to improve their research discussion, the formal presentation of their work, 
and ensure that previous work is recognized and cited (Mark Ware Consulting Ltd 2008, Brown 2009). 

12 A proliferation of new, online-only journals has made it “too easy to publish” in some fields, such as history. 

There is a strong aversion to a “glut” of what is perceived to be loosely-vetted publications (Harley el al. 2010). 

13 Consequently, membership in an academic community is one of the most important incentives for peer referees 
(Mark Ware Consulting Ltd 2008, Brown 2009). For example, the “lists” of work published by university presses in 
a subfield, often under the auspices of a particular scholar-editor, are highly important frames for scholarship and 
progress in the field or subfield. 

14 Time from initial submission to publication can be six years in musicology and five to ten years in economics 
(Ellison 2002). 

15 Junior faculty, in particular, can be in real trouble if they submit to the wrong journal and must wait even longer 
for publication. 

16 See for example, a recent Associated Press article on the problem of plagiarism in China (by Gillian Wong, 
04.10.10, “Rampant cheating hurts China’s research ambitions”): 

http://www.google.com/hostednews/ap/article/ALeqM5iQlSG wOnzcnKwwCZ10Q68WNfNqgD9F0KTVQ3 

17 The NIH’s Transformative R01 Grant is one attempt to counter this tendency. In political science, some scholars 
have called for new scholarly publications for provocative work (Harley el al. 2010). 

IS [KY, peer review meeting, 03.05.2009]. 

19 While double -blind peer review guards the reliability of peer review and protects women and foreign scholars 
(Guterman 2005), it is likely that reviewers could discern the identity of an author through a simple Google search. 

20 [AE, peer review meeting, 03.05.2009]. 

21 In archaeology, one scholar-editor reported that most referees tend towards “description rather than analysis” 
(Harley et al. 2010). 

22 [NJ, peer review meeting, 03.05.2009]. 

23 In particular, scholars in the sciences have complained about “failed scientists” or “teenagers gone wild” who are 
unable to judge the academic quality of submitted work or referee feedback (Harley et al. 2010). Similar complaints 
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concerned the perception that the top general interest journals have “taste committees” that make highly subjective 
decisions to publish work in certain “hot areas.” 

24 Examples include the 2006 retraction of Woo Suk Hwang’s research on stem cells in Science, and the controversy 
surrounding the archival data presented in historian Michael A. Bellesile’s Arming America: The Origins of a 
National Gun Culture. 

25 Consequently, there is an increased reliance on the publication of data sets, images, and evidence supporting an 
argument. 

26 [CL, personal communication (following participation in a 2010 meeting on Electronic Publication convened by 
the National Academies in Washington, D.C.), 03.31.2010). 

27 The recent “ClimateGate” scandal is a particularly good example of the political uproar that can result when 
internal discussions among scholars and questions over the legitimacy of scientific data move into the political 
arena. See, for example, a recent article in Science Insider (by Eli Kintisch, 12.02.09, “Holdren, Lubchenco On 
Defensive About ClimateGate at Hearing”): http://news.sciencemag.org/scienceinsider/20Q9/12/holdren-lubchen- 
1 .html 

2S Alberts et al. (2008) note that the current system is important, but must be improved by making little quality 
tweaks. Researchers want to improve, not replace traditional peer review (Brown 2009). 

29 Bepress features an “Author and Reviewer’s Bank” in which submitting authors can agree to do two reviews 
themselves in a timely manner (or pay $350) to ensure the quick peer review of their own submission. 

30 The Empirical Musicology Review publishes the first two out of the three peer reviews it receives. This 
incentivizes referees to return work quickly by giving then publication credit for their work. 

31 In the bepress family of journals, authors actually submit to the multiple tiers of journal at once, and following 
peer review, a scholar-editor determines where the paper would be best suited. Molecular Biology of the Cell also 
allows authors to forward unaltered editors’ and reviewers’ letters from previous submissions for consideration in 
their review process. 

32 At the journal Plant Signaling and Behavior, authors are required to address the issues of one reviewer before the 
paper is sent to a second reviewer; this allows the second reviewer to focus on more substantive issues, and 
eliminates redundant work (Lev-Yadun 2008). 

33 Although commercial and society publishers dominate journal publication, many society journals contract out 
their publication process to professional publishers (which can include university presses or commercial publishers) 
(e.g., Glenn 2008). 

34 A recent study shows that the average cost of journals by discipline is rising dramatically and is particularly 
prevalent in the sciences (journals in chemistry and biology are the most expensive), followed by social sciences and 
arts/humanities. Moreover, research shows that, on average, libraries pay four to six times as much per page for 
journals owned by commercial publishers as they do for journals owned by their non-profit society counterparts 
(Noll and Steinmuller 1992, Bergstrom and Bergstrom 2004, Edlin and Rubinfeld 2004, Nevo et al. 2005, Van 
Orsdel and Born 2009). In the meantime, for-profit journals continue to enjoy rising profits. 

35 Of course, if the same referee is asked to review a manuscript again for another outlet, s/he may choose to simply 
copy and paste the earlier review. Therefore, the cost in researcher time for referees to review manuscripts that have 
been rejected and resubmitted to other outlets may be less than projected. 

36 Scholar editors at smaller not-for-profit journals, particularly in the humanities, may receive little support and 
enlist graduate students or other assistants on a volunteer basis. In some cases, journals will publish “special issues,” 
which are a way for editors of the journal to bypass the formal peer-review process and delegate the tasks of locating 
articles to guest editors (Abbott 2008). 

37 [AE, peer review meeting, 03.05.2009]. 

38 Plate tectonics is an old example of delayed emergency; the research that led to bacterial causes of stomach ulcers 
is more recent [DS, personal communication, 03.20.2010]. 

39 New article-level metrics have not yet replaced traditional citation indices or the impact factor in tenure/promotion 
and grant evaluation processes. 

40 (For well-established authors in some fields, such as economics, repositories may improve their ability to 
distribute work outside of the traditional peer-review process, Ellison 2007. It is important to note that such practices 
by younger scholars at competitive institutions are probably quite rare.) 

41 See, for instance, the model proposed by Ginsparg (1996) for electronic research communication in physics . 

42 Lightly or non-peer-reviewed conference proceedings may enable scholars to disseminate their work in 
humanities fields that have long lags to monograph publication. 
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h The review process, which is highly competitive for the most select outlets (such as the Association of Computing 
Machinery) ensures the visibility of the best papers (Casati et al. 2007). None of the 12 fields studied in Harley et al. 
(2010) was observed to be moving to the computer science model. 

44 The blind review process is said to ensure the reliability of published work that can pass the test of anonymous 
reviewers. 

45 Nature experimented with an open online peer review trail from June to December, 2006 (cf. Nature 2006). 

46 See those sponsored by the Institute of the Future of the Book: http://www.futureofthebook.org/ 

47 Even Wikipedia is beginning to limit who can change articles by instilling a form of “peer review” for featured 
articles, see: http://en.wikipedia.Org/wiki/Wikipedia:Peer review 

48 For instance, scholars can use software such as Publish or Perish . 

49 More recently, the eigenfactor , another journal-level metric, rates a journal’s importance. 

50 For a comprehensive review of article-level metrics, see http://article-level-metrics.plos.org/ 

51 This can include commentator prestige and the nature of the language used in comments (positive, negative, 
clarified, etc), among others (Jensen 2006). 

52 Some services, like ISI or Scopus, are subscription-based and expensive. 

53 See also Citemine based on a new metric of what is worth reading by integrating a bidding system into OA 
repositories. Developed from an economic model, bids represent user judgments of a paper's promise in attracting 
future citations. 

54 In addition, an item’s location in Google or Google Scholar provides us with some sort of evaluative measure 
(note GPeerReview by Google is in development). 

55 For instance, the working paper repository SSRN ranks authors and papers by the number of downloads providing 
an informal quantitative indicator of the value of a working paper. 

56 Although there is some evidence of blog-based peer review prior to final archival publication, blogs, as a source 
of scholarship, are off the radar for many scholars because they are not formally vetted. Our work suggests that 
blogs are also not a common way in which scholars publish in-progress or archival research (Harley et al. 2010). 

57 Efforts like the FI 000 may be able to reduce the pressures faced by young scientists, who often place undue 
emphasis on publishing in a few high-profile journals (even when the criteria used for evaluation may not match 
their research) (Alberts et al. 2008). F1000 might be another way to value the contributions of young scientists who 
publish outside of these few journals. 

58 The h-index places the emphasis on the author by attempting to measure the author’s scientific productivity and 
impact. 

59 This index spans a scholar’s productivity including publications, citations, federal research funding, awards and 
honors. 

60 As some have shown (c.f., Bollen et al. 2009; RIN 2009), different measures of citation and usage log data 
express scientific impact in different ways. In particular, Bollen et al. (2009) found that impact factor is actually one 
of the least representative citation measures. 

61 For more information on the controversy surrounding reliance on the impact factor see, e.g., Monastersky (2005), 
Altbach (2006), The PloS Medicine Editors (2006), Brown (2007), Hobbs (2007), Williams (2007). Other criticisms 
have been levied at citation statistics writ large, including by Howard (2008b) and Greenberg (2009). Also note the 
upcoming NSF workshop on “Scholarly Evaluation metrics”: 
http://informatics.indiana.edu/scholmet09/announcement.html 

62 It is clear that any new peer review system must be able to reliably predict the significance of a piece of work, 
produce an easily digestible recommendation to help people decide what to read, be economical in terms of reviewer 
time, work quickly, and be resistant to gaming by authors (Jennings 2006). 

63 Other studies of faculty norms and values have come to the same conclusion, including Weale et al. (2007). 

64 This discussion pertains only to “royalty-free” work, such as not-for-profit scientific communications, which 
predominantly takes article form. Book-length publications create different considerations outside of this model, and 
experiments with open access book publication are described by Adema (2010). 

65 OA does not require authors to waive their rights under copyright law. As Suber (2007) describes, when authors 
consent to OA, they usually consent to the unrestricted reading, downloading, sharing, storing, printing, searching, 
linking, and crawling of the full-text, but often block plagiarism, misrepresentation, and commercial re-use. 

66 For example. Cell Press’s author-rights policy allows authors to post a “revised personal version” of the final 
article text and illustrations on a personal website with a link (via DOI) to its published location. 
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67 Creating Green OA to the final version of record is preferable, but, while most publishers are relaxing their 
policies on the posting of preprints, they are tightening their embargoes on final accepted versions (Cox and Cox 
2008). 

68 In January, 2010, the Directory of Open Access Journals (DOAJ) listed a total of 4,570 peer-reviewed OA 
journals. A notable publisher of OA journals is BioMed Central , launched in 2000 and recently bought by Springer. 

69 Universities UK (UUK) and the Research Information Network (RIN) recommend that funding bodies and 
universities pay publication fees on behalf of their grantees and faculty. 

70 In theory, the extreme increases in some journal prices and their resulting very high price levels would cause 
universities marginally committed to research in disciplines covered by those journals to drop subscriptions in a 
public access world (where much of the material in journals can be obtained for free after an embargo period). In 
markets it takes only a small number of buyers responding to price stimuli to reduce equilibrium prices. It is for this 
reason that many commercial publishers continue to oppose public access mandates [DS, personal communication, 
3.20.2010], 

71 As Suber (2010) outlines, there are several European-wide policies in development for open access, including the 
European Research Council’s mandate for Green OA following a 6-month embargo. The UK is clearly the country 
with the greatest number of agencies mandating OA to publicly funded research; six of the seven Research Councils 
UK now have adopted mandates, and the seventh (Engineering & Physical Sciences) is still deliberating (Suber 
2008). The Wellcome Trust also stipulates open-access dissemination of research results as a condition of funding. 

72 The Fair Copyright in Research Works Act was introduced to prevent federal agencies from mandating transfer of 
copyright or limiting the exercise of copyrights, but has mobilized little support (Howard 2008a). Instead, 2009 saw 
the re-introduction of the Federal Research Public Access Act (FRPAA), which would mandate public to research 
results within six months after publication for all the major US federal funding agencies (cf. Suber 2010). 

73 This mandate follows a looser “request” for NIH investigators to deposit work in May, 2005. The NIH Public 
Access Policy is available at: http://publicaccess .nih. gov/ 

74 For example, the Nature family of journals began offering to deposit articles automatically into PubMed in the 
summer of 2008. The archived versions go public after a six month embargo (Guterman 2008). 

75 See the agreement at: http://www.hhmi.org/news/hhmielsevier20070308.html 

76 Since 2001, the American Society for Cell Biology (ASCB 2009) has provided access to scientific articles 
published in Molecular Biology of the Cell two months after publication, both on the journal’s website and in 
PubMed Central. 

77 The American Association for the Advancement of Science allows author manuscripts of research articles it 
publishes to be posted elsewhere six months after publication. 

78 The first university OA mandate was in 2003 and the numbers have grown each year. 60 were adopted in 2009, 
(Suber 2010), and more universities are following with planning commissions and recommendations to their faculty, 
such as Indiana University (cf. Wheeler and Acito 2009). 

79 Peter Suber tracks university OA mandates at: http://www.earlham.edu/~peters/fos/fosblog.html 

80 Over 860 IRs are currently reported by the Registry of Open Access Repositories (Schmitz 2008, Fry et al. 2009). 

81 The Harvard Arts and Sciences Faculty (ASF) voted unanimously (02/12/08) to adopt an open access policy. It 
was followed by Harvard Faw School (05/01/08), and the Harvard Kennedy School of Government (03/1/09). The 
ASF policy is online at http://www.fas.harvard.edu/~secfas/February 2008 Agenda.pdf ; see also Stuart Shieber’s 
blog: http://blogs.law.harvard.edu/pamphlet/ 

82 See the policy at: http://info-libraries.mit.edu/scholarly/faculty-and-researchers/mit-faculty-open-access-policy/ 

8 ’ Harvard University and the American Physical Society (APS) entered into an agreement whereby the APS will 
recognize Harvard's open access license without requiring copyright agreement addenda or waivers from faculty 
authors: http://4sustainabilitv.blogspot.com/2009/04/harvard-and-aps-reach-accord-on-iournal.html 

84 In one possible approach, the NIH has decided to “consider” direct feeds from IRs to PubMed Central, but the IR 
would assume responsibility for the deposit (See Suber 2010). 

85 The final resolution has an “opt in” clause and is available here: 

http://www.virginia.edu/facultvsenate/documents/FacultvSenateResolutionrevised 9.09meeting 003.pdf 

86 Many of these U.S. universities have signed the Compact for Open-Access Publishing Equity (COPE). 

87 The University of California, for instance, has negotiated with Springer for its subscription payments to cover the 
publication fees for UC authors to publish in Springer's hybrid OA journals. 

88 Open access or copyright workshops led by librarians or fellow scholars have proved helpful at some institutions. 

89 Although 60% is much higher than the 4% voluntary deposit rate prior to the mandate, and the deposit rate 
continues to grow, it is still not 100%. 
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90 This is similar to the arguments made in support of MIT’s Open Courseware. 

91 Their access difficulty increases as journals go to electronic form and the interlibrary lending that is permitted 
under the copyright laws cannot produce access for them. 

92 As Swan (2010) found (using a very specific cost model), the savings generated by moving from subscription- 
only models to IRs or per-article OA charges varies by university (depending on size and author fees); savings 
ranged from $500,000 at a small school to $8 million at a highly-competitive research-intensive institution. 

92 Scholarly communication needs include the complaints about the peer-review and publication process presented 
in Working Paper 1, as well as a shortage of professional publication outlets for many genres of work like 
monographs, digitally innovative scholarship, cross-disciplinary work, and long-form articles (Harley et al. 2010). 

94 According to Kling and Spector (2004), the strength of any scholarly publication can be judged according to three 
criteria: trustworthiness (quality indicators, such as peer-review, publisher quality, and sponsorship), publicity 
(making relevant audiences aware of the publication), and accessibility (it can be readily located and obtained). 

95 For example, some scholarly societies have launched new journals to overcome long lag times (see, for instance, 
the four new online journals launched by the American Economic Association in 2009), and others are extending 
their publication remit to include new multimedia-based digital monographs based on an on-demand or OA model 
(e.g., APA/AIA Task Force on Electronic Publications 2007). 

96 For example. Nineteenth-century Scholarship Online (or NINES ) is a scholarly organization that oversees the 
peer-review of digital scholarship and products in this subfield, and subsequent inclusion in the NINES repository. 

97 The need for widespread digital preservation has also led to services, such as the Depot , which enables researchers 
without an institutional repository to deposit their work. 

98 For example, the arXiv preprint repository is used as a locus of submission by some scholarly society publishers. 
After receiving an author alert to relevant content, these publishers must download the relevant preprints from the 
repository and then provide the peer review and publication process (Swan 2010). Future work at the arXiv may see 
this submission process become more formalized and streamlined. 

99 Rodriguez (2006) presents a publication model built around an Open Archives Initiative Protocol for Metadata 
Harvesting (OAI-PMH) peer-review service. This service draws on a social-network algorithm to locate relevant 
reviewers for a submission as well as weight their subsequent evaluations. 

100 As Shulenburger (2007) describes, in the humanities, monographic work that cannot get published due to press 
budget limitations could be vetted by one’s scholarly society and, in time, entered into a series distributed by an IR. 
In this case, the publication would carry the imprimatur of the scholarly society that organized the vetting. 

101 For example. The University of California Publications in Entomology is a monograph series published by UC 
Publications, a relation of the University of California Press, that is digitally published in the UC’s IR: eScholarship. 

102 Scholarly output needs to be tagged by metadata (including title, creator, subject, description, publisher, rights, 
etc) (Canadian Federation for the Humanities and Social Sciences, 2006). The Open Archives Initiative (OAI) has a 
protocol for metadata interoperability (Schmitz 2008). 

103 Ivins and Luther (2009) observe that some things that are important to libraries are not important to small or 
scholar-run journals, such as providing institutional subscription access, issuing renewal notices, creating digital 
access; correspondingly, small journals may be more interested in visibility or maintaining a specialized community 
than revenue or managing efficient decision chains. 

104 IRs can be built on numerous platforms (including DSpace, Fedora, Digital Commons, and Eprints). Foster and 
Gibbons (2005) research shows that with a “faculty-centric approach to the design and marketing of repositories, IRs 
could become a compelling and useful tool.” Research evaluating limited faculty uptake at Cornell found a lack of 
DSpace functionality using Cornell’s DSpace (Davis and Connolly 2007). 

105 For example, the Mellon-funded American Literatures Initiative joined university presses at NYU, Fordham, 
Rutgers, Temple, and Virginia to secure, publish, and market first monographs in the field. In particular, the 
initiative established a shared, centralized editorial service dedicated to managing the production and preparation of 
books in the initiative: www.americanlitemtures.org (University Presses Collaborate 2008). 

106 In particular, there is a need among scholars for shorter “monographs” in some humanistic disciplines and longer 
articles in the sciences (notably biology) (Harley el al. 2010). University press may be well-positioned to experiment 
with new publication genres/provide digital research and publishing platforms. For instance, in 2006, Rice 
University Press was reborn as “the nation’s first all-digital academic press.” 

107 For example, Columbia International Affairs Online ( Cl AO ) not only publishes peer-reviewed work in 
international affairs, but is also a resource for working papers, policy briefs, and teaching materials (although it may 
be becoming outdated as scholars turn to Google to search personal websites rather than resource portals). 
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108 For example, bepress “develops and licenses technologies to help the academic community produce, archive, and 
disseminate scholarly work.” 

109 See Hilton’s argument in Harley (2008). 

1 10 For an example of how the arXiv funding model is changing to reflect library community contributions, rather 
than subscriptions, see Davis (2010). 
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